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Sharing the Data along with the Responsibility: 

Examining an Analytic Scale-based Model for Assessing School Climate 

Abstract 

This study reports the findings of a pilot effort to examine the efficacy of an 
analytic trait scale school climate assessment instrument and democratic change 
system in two urban high schools. Pilot study results indicate that the instrument 
showed promising soundness as it exhibited high levels of both validity and 
reliability. In addition, the analytic trait format demonstrated the capacity to 
promote deeper levels of participant reflection and encourage greater participant 
empowerment and better process efficacy than its objective survey counterparts. 
Data analysis suggested a 4-quadrant model for examining “climate goal 
orientation” (included in the paper). Given what appeared to be an inherently 
socially constructed quality to each school’s collective definition of what they 
considered “good school climate,” the use of the analytic instrument and a 
transparent assessment process proved useful. Moreover, the participant-driven 
feature of the system demonstrated a high degree of efficacy in helping 
participants move effectively from the climate assessment phase to subsequent 
vision setting and action phases. Findings related to the climate and assessment 
process at the two schools as well as overall study conclusions and implications 
for practice are offered in the paper. 



O 

ERIC 



Sharing the Data along with the Responsibility 



2 



3 



WASSC-AERA: 2003 



Sharing the Data along with the Responsibility: 

Examining an Analytic Scale-based Model for Assessing School Climate 

As schools seek ways to improve educational quality, the search often leads to an 
examination of their climates. While school climate improvement appears to have taken 
a back seat to more “direct” efforts to effect academic performance, evidence persists 
that initiatives for improving school climate can positively impact student learning 
(Bulache and Malone, 1994; Freiberg, 1999; Hoy & Hannum, 1997; Keefe and Keller, 
1990; Stewart, Evans and Kaczynski 1997). Moreover, because school climate is 
predictive of many other indexes of school effectiveness, it is a fertile arena to explore. 
Many of the critical issues that face schools today are related to climate (Bobbett, 1991; 
Bulache, 1994; Nidiche and Nidiche, 1986). No wonder that so much has been invested 
into assessing school climate. One indication of this investment is the number of school 
climate and/or culture inventories. In a 1987 Report (ERIC Reports: Northwest Regional 
Educational Laboratory), 42 separate school climate inventories were identified. Since 
that time other school climate inventories have also emerged (Bernardo, 1997; Bobbett 
& French; Crawford and Irwin, 2002, Butler and Rakow, 1995; Haynes, Emmons and 
Comer, 1994; Jones, 1996; Olafson, Bendixen and Tirella, 2002; Worrell, 2000). Given 
the shear number of these instruments and the diversity of their representations of 
“climate,” there appears to be a need to clarify the basic theoretical conceptions 
underlying the domain of school climate. Moreover, there are fundamental concerns 
with the efficacy of any school climate improvement system featuring a survey type 
inventory driven by university researchers, given that the school stakeholders are often 
not meaningfully involved in the assessment process (Fullan, 1992; Zeichner, 2002). 

The Pilot Study of the WASSC Assessment System 

The paper reports the results of a pilot examination conducted by the Western Alliance 
for the Study of School Climate (WASSC) of an analytic scale-based system for 
assessing school climate in two urban high schools. The eight-factor analytic trait 
instrument piloted in the study was derived from school climate factors identified by 
previous research into high functioning schools as well as theoretical relationships 
among the eight factors. The assessment instrument under investigation was part of an 
overall system for facilitating a participant-led democratic climate reform process. This 
proposed process would include an initial phase related to reflection and self- 
assessment, a second phase for democratic vision setting, a third phase for strategic 
planning, followed by a fourth phase characterized by faculty-led plan implementation. 
This process is consistent with those of other research-based reform frameworks 
(Fullan, 1992; Marriott, 2001). The foremost purposes of the pilot effort were to validate 
the soundness of the instrument as well the broader system for school change. The 
study offers findings related to the system validation process as well as the process of 
school climate assessment within two schools. 

A Theoretical Comparison 6f Assessment Instruments and Systems 

The prevailing model for assessing school climate involves the use of survey-type 
objective inventories, which are administered and/or scored by outside agents. These 
objective surveys are often given to parents, teaches, and/or students from which a form 
of numerical “school climate index” is derived. This approach to assessing school 
climate is typically driven by educational researchers who conceive their task within a 
quantitative framework. The advantage of such a model is its potential ability to make 
comparisons of statistical significance among groups or schools. 
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The alternative model examined here, and developed by the researchers who formed 
the Western Alliance for the Study of School Climate at California State University Los 
Angeles in 2001, utilizes an anal^ic trait scale instrument for use by a committee of 
stakeholders assisted by university partners. Analytic trait type instruments have been 
shown to have several advantages over surveys, have the capacity to be more sound 
than objective surveys (Shindler, 2002; Stiggins, 2001), and are gaining wider use, 
especially in the area of program assessment (Henderson and Hawthorne, 1995). One 
of the significant benefits of using analytic school climate instruments may be thier 
capacity to provide their users with a basic construct for defining good school climate, a 
capacity virtually absent in objective survey instruments. 

The two survey items shown in Figure 1 illustrate the differences in item construction 
between the two types of instruments. Item A is taken from an objective survey type 
format instrument whereas item B is taken from the revised version of the analytic trait 
scale type instruments examined in the study. 

Figure 1: Comparison of Assessment Instrument Item Formats 

Item A: Objective Survey Item Example: 

Teachers at my school help us children with our school problems 

Agree Not Sure Disagree 



Item B: Analytic Trait Scale Instrument Item Example 



Teacher-student interactions could 
be typically described as supportive 
and respectful. 


Teacher-Student interactions could be 
typically described as fair but teacher- 
dominated. 


Teacher-student interactions are 
mostly teacher-dominated and 
reactive. 










Methodology 





Data were collected in four phases. First, in each school, the principal selected a 
convenience sample of teacher participants to act as the pilot “steering/assessment 
committee.” (N=6 and 7). In each case the sample consisted of teachers who shared a 
particular planning period. This committee was interviewed regarding overall perceptions 
related to the state of affairs at their school and were given version 1 of the instrument to 
examine and make suggestions. After receiving participant feedback, revisions were 
made to the instrument items and design. Second, the revised analytic instrument (see 
figure 1 for sample items) was re-administered to the teacher participants, and two 
classes of student participants at each school, representatively sampled from the school 
as a whole (N=51 and 57). After administration of inventories, participants were 
interviewed using a semi-structured focus group technique. Third, research team 
members made independent assessments of the school in each area of the instrument. 
Data for this external evaluation included classroom observations, informal interviews 
with student and faculty participants, participant-observation from steering committee 
meetings, and observation of day to day school life. Fourth, teacher participants were 
interviewed regarding their perceptions of the potential efficacy of the instrument and the 
system as a whole. Data were analyzed using a constant comparative method. A 
grounded theoretical framework for analysis was developed (see Figure 2). 
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Study Findings 



Soundness of the Analytic Scale-based Instrument 

The pilot version of the analytic trait instrument showed promising soundness as it was 
administered to teacher participants at each school. Subsequent analysis of soundness 
was done for the revised instrument (version 2) in the areas of validity, reliability, efficacy 
and usability. 

An examination of the validity of the instrument demonstrated four findings. First, after 
the initial administration of the instrument (version 1), teacher participants reported that 
the items were relating to “actual school reality,” and “made sense.” While they had 
minor suggestions for item changes, overall they felt that the items depicted a genuine 
picture of school reality. This appraisal suggested a sufTicient level of what could be 
considered content and/or face validity. 

Second, participants, given the revised instrument (version 2), reported seeing the 
relationship among items in each sub-scale area, showing evidence of sub-scale 
construct integrity. Moreover, their ratings for items within each sub-scale tended to 
show a great deal of consistency. This integrity could be seen further in the reliability of 
mean ratings of items in each sub-scale between teacher and student participants. 

Third, further construct validity was observed in version 2 in the degree to which its sub- 
scales scores assessments inter-related. Most participants suggested that their overall 
ratings for any one sub-scale were similar to their ratings for the others. This consistency 
was confirmed by the high correlation among participant’s sub-scale mean ratings. 
Notwithstanding the high degree of rating consistency, the intention of such an 
assessment instrument is for some independence of factors. An analogy might be that of 
a race among 8 runners that are all linked by ropes. There will be a first and last place 
finisher, but in the end all of the racers will finish within a limited range. In other words, a 
school may perform better or poorer on any particular factor, yet the performance on any 
single factor will always be buoyed or pulled down by the location of the mean of all 
factors collectively. This finding suggests that sub-scale domains may be theoretically 
independent, yet the practical interdependence of any teaching practice or aspect of 
school life to another appears to be evident. 

Fourth, as participants and researchers examined the 3 “levels” in the instrument, a 
theoretical construct emerged for each. This was first observed when participants often 
noted that the “3 level” was impossible or could not be achieved in their urban school. 
Participants at the lower performing school suggested that with the students they had, “3 
level” teaching and climate were beyond their reach. This sentiment reflected the 
identification of a “type” of school that was being characterized at each level. This 
analysis led to an attempt at conceiving these levels within a theoretical framework to 
explain each school “type” (Figure 3). This appearance of a relationship between item 
level and something that could be considered school “type” demonstrated further 
strength of the fundamental constructs of the instrument. While the classification of any 
particular school into a type is not the intention of the instrument or the assessment 
exercise as a whole, there seems to be some practical value to the existence of patterns 
among the levels. 
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The reliability of the pilot instrument was seen as important because it demonstrated that 
the instrument depicted commonly understood reality and articulated that reality 
accurately. However, variation in each participant’s climate ratings would be assumed 
due to differences in perceptions. 

Overall, most items demonstrated acceptable inter-rater reliability, as shown by a strong 
correlation among student, teacher and researcher ratings. From data collected in the 
focus group interviews, it appeared that most of the variance in ratings was due to 
subjectivity in perceptions of the reality that existed at each school, and little seemed to 
come from a disparity in item interpretation. This finding supports the theoretical 
assumption that an analytic scale assessment instrument would promote a high degree 
of inter-rater reliability due to the concreteness and clarity of its structure (Stiggins, 2001 , 
Shindler, 2002). In addition, the concrete language provided by the three descriptive 
options (versus an objective survey item with a agree of disagree choice from a single 
description, see Figure 1), provided participants a better conceptual anchor, improving 
intra-rater reliability. Figure 2 illustrates the language used in the revised instrument. 

After administering the pilot version (version 1) of the instrument, interviewing 
participants and analyzing the phase 1 data, the following changes were made to the 
revised version (version 2) to improve its technical performance and theoretical 
soundness. First, language was changed to make items clearer, more universal and 
inclusive of regional vocabulary and jargon. Second, some items were improved to 
better create a parallel structure and better reflect a continuum of practice/behavior. 
Third, items were added that were seen as missing elements of each of the eight factor 
constructs. Fourth, given that each item reflected a continuum of practice, it was 
determined that giving participants three options per descriptor provided an opportunity 
to give “high,” “middle,” or “low” ratings for each. As a result, each item could be scored 
on a nine-point scale. This change provided better performance level discrimination. 

And, while it may have weakened the theoretical reliability of the items, this was offset by 
improved consequential validity. That is to say that it better provided participant raters 
the perception that they were rating practice/performance across a range or continuum. 
This benefit would prove additionally useful later in the process of application as 
participants attempted to diagnose where changes were necessary. In this planning 
phase, participants, given their ratings in front of them, instead of asking “which 
descriptor best characterizes where we are?” were more inclined to ask, “where do we 
currently fall on this continuum?” 
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Figure 2: Sample items from scales 2 and 7 from revised analytic instrument: 



2. Faculty Relations . 
■High 


:-i?Middle , /Itdw? • 


■ IHigh; " Middle-/. Low 


Faculty commonly collaborate on 
matters of teaching 


Faculty are congenial to one another, 
and occasionally collaborate 


Faculty see other faculty as the 
competition. 


< — ^n-“r- — n — n — -- 


• -r ^ • n :::-M - - -n -- 


-«n -S'fc: 


Faculty approach problems as a 
team/collective 


Faculty attend to problems as they 
relate to their own interests. 




Faculty expect someone else to 
solve problems. 


n - : 













7. Attitude and Culture ,y; Vf: ' ■ • 

:y^i:Xeye/-3ia Leyhl-2 <-4|fg. : , .LevelP:! ■-?,-# 

^Middle ■ jjLow r^»afe»--«Hl qh -'^Middle! ^4SLbw|:---V ' Mid<BdllgUow%» 

Students feel like they are part of a Students feel like they are part of a Students feel like they are visitors in a 

community. society. building. 

<„.n— — — n — — - n — n — — - n — — — na-- — - — n — n — n — SI 

Students self-correct peers who Students seek adult assistance to Students accept abuse as a regular 

use destructive and/or abusive stop blatant abuse. part of their day. 

language. 

<— — n — n — — n - — 



The revised instrument proved to be sufficiently user friendly. Both teacher and student 
participants were able to complete the 73 item ratings in about 20-30 minutes. Only 
minimal instructions were required relate to the “high,” “middle” and “low” classifications. 
Moreover, the analytic trait instrument demonstrated high levels of “usability” within the 
context of the vision-setting phase. The analytic instrument provided participants in each 
school not only a sound diagnostic instrument, but also a basis for initiating a discussion 
about prescriptions for improvement. As participants noted where they were on the 
continuum for each item, they could be seen taking advantage of the conceptual and 
practical language the items implied for what might constitute improved performance. 



Functioning of the Participant-Driven Assessment Process 
Considering that the “planning phase” of the process for teacher participants was 
undertaken as a “hypothetical” exercise, the process itself showed the promise of 
effectiveness. Participants agreed strongly with the contention that assessment data in 
the hands of the users would help promote a better and more enduring set of outcomes. 
And while the exercise was done within a pilot context, each set of teacher participants 
grew in the sense of ownership of their respective processes. 

Recall that each group of teacher participants was selected for convenience. This quasi- 
random sampling method was useful for purposes of assessing the efficacy of the 
instrument and obtaining a representative sense of the faculty attitudes in general, but 
create limitations for a genuine change effort. As the efforts progressed, each of the 
school’s principals questioned whether the same teachers who took part in the 
assessment phase of the effort would be the best ones to be involved in the planning 
phase. 

A more purposeful selection of team members seemed necessary if the efforts were to 
be fully successful. While the representative composition of the participants was a 
benefit, there appeared to be qualities that would be requisite in those that could 
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effectively guide the next phases of the process. Some of the necessary qualities 
identified by the principal of the high performing school included a commitment to the 
effort, a visionary orientation, and some political sensitivity and/or political capital. While, 
both principals concluded the members of the faculty group were in many cases not 
those that they would have chosen if the task had involved an implementation phase, 
enlisting an entirely new team for a subsequent phase of the process would prove 
problematic. Due to the significance of the learning curve for participants who had gone 
through the process of acquainting themselves with the “state of the school climate” 
while taking part in the assessment phase of the process, new members may not be 
able to duplicate this immersion into the process of determining the starting point for 
planning and taking action. Therefore, it appears that a thoughtful selection process of 
vision committee members at the beginning of the process would be desirable for the 
coherence and the long-term efficacy of the effort 

The response of each school to the opportunities provided by the assessment pilot 
efforts varied somewhat. In the low performing school, the exercise was greeted with a 
sense of hope. These participants reported seeing a series of new programs come and 
go over the years with few seeming to systemically address the school’s more 
substantive needs. They could see the validity of an effort driven by faculty and guided 
by a more democratically derived vision. They saw the assessment instrument providing 
the beginnings of a road map for their reforms. When asked how much of a role the 
university team should play in the implementation phase of the process, these 
participants suggested it should be about 70% school participant-driven and 30% 
university partner-assisted. 

Participants in the high performing school shared the perception of the need for the 
process to be driven primarily by the school team, however, their perceptions of the pilot 
process varied because they did not feel the same degree of need for change. Like the 
low performing school’s teachers, they saw the validity of the democratic process feature 
of the system, and thought that the instrument would be useful in their efforts to 
formulate plans for change, but were not as emotionally committed. 

Academically the two sets of participants provided similar insights, largely supportive of 
the components of the pilot system, yet the school that perceived a greater need showed 
an eagerness to “do it for real,” while the other school felt relatively comfortable with 
where they were going. These teachers seemed not to move beyond an academic 
engagement in the process. However at the low performing school there was evidence 
of emotional buy-in to the process. For example, when the principal at that school 
publicly made the decision to discontinue the process after the assessment phase, the 
teacher participants were visibly disheartened. They felt an opportunity was missed. In 
contrast, a vision and ethos of change appeared to be in place at the high performing 
school, which may explain what appeared to be a less enthusiasm for participating in the 
school climate assessment effort. 

Assessing Climate; Common Findings from Plot-Schools 

Several findings were suggested from an analysis of the data regarding the quality of 
school climate at each of the two schools. Many were unique to each school in 
particular, but four were common for each and therefore could be considered valuable in 
examining the nature of the process itself. 
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First, the school climate experienced by students in various academic “tracks” at each 
school seemed to vary significantly. Both teacher and students participants suggested 
that there were essentially three climates at each school - the advanced placement 
track, the regular track and the basic track. The methods used and the experiences of 
the students in each track were rated at different locations on the continuum, with the 
advanced placement track being the highest and the basic track being the lowest. 

Second, an analysis of the data from the focus groups and the process of survey 
revision demonstrated the “socially constructed” aspect of defining any “good school 
climate.” Participants agreed that a questionnaire style survey is more likely to mask this 
element. With an analytic trait type instrument there is a relative transparency to the 
constructs. Instead of making the assumption of an “a priori” definition, examining the 
notion of what makes for good school climate appears to have benefits for teachers 
undertaking an effort to first diagnose their school’s ills and then prescribe remedies. 

Third, this socially constructed nature to the concept of “good school climate” could be 
seen clearly as participants examined the “level 3” characterization of school climate in 
the instrument and found that the descriptions of practices to be in many ways unfamiliar 
not just relative to what they were doing, but what they were trying to do. The process of 
reconciling these level 3 descriptors with their own situation, needs, capabilities, and 
goals, proved useful but at times troubling to many participants. For example, after 
completing the initial version of the survey and being asked for their impressions, one 
participant’s response was, “My thought was that now I need to change everything I do.” 
One could conclude from this statement the goal assumptions he held were in conflict 
with the implicit goal logic reflected in the instrument. This conflict could be seen in many 
of the participants as they rated their schools 5 or 6 out of a possible 9, in areas where 
they had reported being rather satisfied with their own performance or the performance 
of the school. Each participant’s and/or school’s definitions of “practices they felt 
promoted a good school climate” could be seen as bound by individual socio-cultural 
values influenced by region, class, ethnicity, religion, and teacher training. 

Fourth, through analyzing the data related to that which participants expressed as their 
“goals” related to school climate, a construct emerged to classify “goal orientation.” 

Figure 2 outlines this two-axis matrix for climate goal orientation. 

Figure 3: Four-quadrant matrix of school “climate goal” orientation 



[KRMBwiil 



DiiQMI®(iDgiQ 



lLih.ej;aiibn/^apjlit A 







^ 'u’©§]@{!D©(r° 




|[ID©fc(W 



Sharing the Data along with the Responsibility 






9 



WASSC - AERA: 2003 



Figure 3 depicts a useful framework for examining the “goal logic” of a particular teacher 
or school as it relates to school climate. The matrix shown in Figure 3 is derived in part 
from the attempts of previous research to classifying teacher practice (Finn, 1999; 
Shindler, Jones, Taylor and Cadenas, 2003), and data from the current study. It reflects 
the intersection of a continuum characterized by two key variables that define goal 
orientation. First, any teacher and/or school practice could be considered more or less 
effective, inasmuch as it produces desirable outcomes. This variable includes the 
component of intentional versus unintentional practice. Data from this study supports 
previous research (Shindler, Jones, Taylor and Cadenas, 2003) suggesting that in the 
absence of intention, practices tended to be either ineffectual or authoritarian. Second, 
any educational practice could be placed on a continuum from more teacher-centered to 
more student-centered. Applying the findings of Finn (1999) the component of 
liberation/facilitative practice vs. domesticating/managerial practice was overlaid upon 
this continuum. The resulting construct provided an addition element to the examination 
related to the “effect’ of practice in contrast to simply examining the nature of the 
practice itself. The juxtaposition of these two axes provides a four-quadrant matrix with 
which to classify school practice and/or school goal orientation. 

A relationship can be seen among the 4 quadrants of the goal orientation matrix and the 
3 levels in the analytic trait assessment instrument used in the study. This relationship 
logic was not employed intentionally in the construction of the instrument, however it 
does seem to be evidenced in an empirical examination of these particular schools. 

Many of the 3 level descriptions for items in the instrument could be characterized by the 
approach defined by quadrant 1 . Quadrant 1 is defined by an intentional student- 
centered approach. Underlying this construct is the value for the presence of community 
and the use of empowering pedagogy. 

Similarly, many of the 2 level descriptions for items in the instrument could best be 
classified by the approach defined in quadrant 2. Quadrant 2 is defined by a successful 
and intentional effort to manage/domesticate students. Underlying this construct is a 
value for an effective societal structure and pedagogy characterized by direct instruction. 
Likewise, most level 1 items characterize practices defined by quadrants 3 and 4. These 
lowest levels could be characterized by a lack of intention and a corresponding 
dysfunctional set of outcomes. These constructs are defined respectively by either an 
absence of structure or an overly controlling environment, and the consequential 
disconnection of students to their schools and one another. 



Interaction of the Assessment Process with the School’s “Implied Climate Goal” 
Vision team participants 

Participants at each school tended to rate their climates somewhere in the level 2 range. 
As discussed earlier, the participants found the process of rating their schools on the 
climate assessment instrument relatively straightforward. However, the ease of rating 
belied the inherent conflict participants experienced in placing their school along the 
continuum of performance depicted by the instrument. While at each school, participants 
could readily accept that the practices and realities depicted in the “level 1” options were 
undesirable, the practices and realities characterized in level 3 brought a degree of 
consternation. 
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Using the paradigm offered in Figure 3 as an analytic lens, what participants seemed to 
be expressing was a conflicted response to the “constructed realities” characterized in 
the items at the 3 level. This reality could be depicted as primarily intentional, liberating, 
and student-centered, and in many ways uncommon in large public high schools. One 
the one hand they could appreciate the worth of such practice on an abstract level, but 
they did not see them as possible or in many cases desirable. At the low performing 
school, the most common reaction to the level 3 item options was to suggest that they 
were not possible with the students they had in their classes, with the possible exception 
of the advanced placement level students. Common initial responses to items at the 3- 
level were “No school could expect all teachers to . . . “ or “that would not work with our 
students." Participants were not asked about their personal goals related their 
“intentional practices” and whether what they saw as “best practice” would fit more into 
the 1 or 2 quadrant in the goal matrix. However, it was clear that many teachers 
suggested that they wanted to achieve good school climate, yet as they discussed the 
practice they believed would help them meet that objective, they often described 2 
quadrant and in some cases, 4 quadrant teacher practice. 

At the high performing school, there was similar perplexity. While the teachers were 
relatively pleased with their climate as they saw it, they also rated their school a high 2 
or upper middle level in most areas on the instrument. Satisfaction with a more 
intentionally managerial/domesticating agenda was more prevalent at this school. Again, 
these participants generally demonstrated less emotional investment, and in most cases 
viewed the gap between their rating and the higher 3 levels on the instrument as 
somewhat academic. It appeared that their goal of school climate defined by 2 quadrant 
practice was acceptable to them, and that what was characterized by the 3 level items/1 
quadrant practice was too idealistic and/or conflicted with some of the more traditional 
values held by the participants. 

At each high school, there seemed to be four common factors that acted to mitigate 
what might have been a more overt level of conflict related to the process, the 
instrument’s implicit “goal logic” or the research team members themselves. First, the 
process was being undertaken as a hypothetical exercise. The stakes were low. 

Second, there was a great deal of effort put forth by the research team to built trust 
among all participants. Great care was taken to provide confidentiality, limit the 
perception of external judgment and assume a posture of listening. Third, while the 
process was undertaken with undefined end-points at each school, care was taken to let 
the participants drive the effort. Last, the process was undertaken with deliberate 
transparency. As each set of participants grappled with where they judged their schools 
to be operating on the climate continuum and how that picture fit with the assumptions 
they had used previously to judge “good school climate,” the fact they had their 
completed instruments in front of them proved valuable. The descriptive language in 
each of the items provided concrete ideas for use in both assessment and planning 
possible next steps. 

Leadership 

The research team gained access to the high schools studied by securing the 
cooperation and participation of the principal. The data yielded by the new instrument 
affirmed some revealing insights about the power and influence of the principal on the 
climate of the school and the scope of reforms open to the faculty. 
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The principal of the low achieving school embraced the pilot study during its early 
phases and provided support for the research team to conduct its visits and 
interviews. However, she remained detached from nearly all interactions between her 
teachers and the research team. Interviews with the principal and analysis of the data 
now shows this principal regarded the pilot study as a routine favor she would grant the 
university because it provided many of her high school’s student teachers. She did not 
consider the possibility that an analytic scale-based instrument to measure her school’s 
climate might uncover problems that require attention. Her teachers, on the other hand, 
regarded the instrument as a description of what their school might become, a center of 
dynamic learning for their students and themselves. 

Near the end of the study, when asked if she would like to work with the research team 
and the teachers she selected to validate the new analytic scale-based instrument to 
then address some of the problems identified, she expressed little interest. When later 
presented the formal findings of the study in a private meeting with one member of the 
research team, she dismissed most of the results as inaccurate. Notwithstanding the fact 
these findings stemmed from data yielded by the new climate instrument, talks with 
students and teachers and several observations of daily routines at the high school 

Applying the matrix of school “climate goal” orientation to this principal suggests she 
perceived herself as an Intentional Climate leader, but one who seeks to domesticate 
and manage her students and teachers. She endorsed the idea of a healthy school 
climate, but only insofar as it serves the purpose of preserving order and a diffuse sense 
of pride in the school. 

As she glimpsed the climate survey’s indicators of a liberating/facilitating intentional 
school climate while hearing findings that suggested passivity and isolation among her 
students and teachers, her dissonance grew. This may have caused her to affirm her 
role as a strong leader of the status quo who rejected findings that might alter this role in 
the future. 

In contrast, the principal of the high achieving high school engaged the research team 
from the beginning. He quickly saw the survey’s indicators of a liberating/facilitating 
intentional climate as a means of detailing the vision he held for his school. However, the 
results of the study thus far suggest the principal’s vision stands far ahead of his 
teachers’ and students’. 

The teachers at the high achieving school took some interest in validating the 
assessment instrument, but none saw it as a means to reform their practice or to change 
the climate of their school. 

At the high achieving school, the principal seems eager to strengthen its climate, but the 
research team has found little desire for improvement among the teachers and students 
interviewed. Their scores on the faculty relations and student interactions items were no 
higher than those at the low achieving school. Teachers seemed to accept their isolation 
as a fact of life and the students expressed little desire for a greater voice in the 
governance and operation of their school. 

The research team will soon confer with the principal of this school to determine what he 
might do next. He knows if he is to succeed he must recruit a new group of teachers to 
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anchor a “school vision team” who possess influence with their colleagues and the 
political savvy to navigate change. 

This illustrates another dimension of the principal’s influence. In the case of the high 
achieving school, the principal perceives his role as an intentional liberating/facilitating 
climate leader who desires a more student-centered culture. To achieve this goal, he 
knows he must use his influence with the vision team of adults (and students, perhaps) 
to alter their beliefs and practices in favor of students. 



Study Conclusions 

1 . The analytic instrument demonstrated promising soundness in this limited 
examination. Further validation is necessary, however it appeared that the 
theoretical assumptions of the study related to the comparative soundness of the 
analytic trait based instrument relative to a survey type instrument were 
supported. 

2. The predicted capacity of the analytic trait type instrument to act as both an 
educational tool as well as an assessment instrument was supported. The 
process of using the instrument as well as the instrument document itself 
provided the internal assessment team a useful construct to diagnose their 
school’s current condition, while providing language for articulating both a system 
for thinking about change and a prescription for action. 

3. Given the differential goals for school climate demonstrated by the participants in 
this study, it might be concluded that no a priori construct for the concept of 
“good school climate” exists, or should assume to exist. School Climate is more 
accurately represented by an ongoing construction undertaken by those who 
work in and interact within a particular school. Any effort to define climate must 
consider its “socially constructed” nature and how that nature might affect the 
implementation of an assessment of a school’s climate. 

4. The assessment instrument under investigation here assessed climate within the 
sociological construct of an “intentional liberating/facilitative” goal orientation. 

This conception is potentially inconsistent with other goals and definitions of 
school climate, including many of those held by participants in the study. 
However, the data here appears to support the idea that the kind of climate 
indicators that were the most desirable to the majority of participants would be 
more achievable with an intentional and student-centered approach. In other 
words, no matter how intentional their efforts, participants would not be able to 
reach level 3 performance using 2 quadrant practices. 

5. We can conclude a significant role for the principal in forming new climates or 
reforming existing ones. No principal of a large and complex high school leads 
unhindered by the actions and influences of teachers, students or the community. 
However, the principal’s role remains critical to improving a school’s climate 
because as the findings suggest, the principal can either inhibit or enable efforts 
to examine the school’s culture and it is chiefly the principal who holds the fate of 
this examination. 
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6. The participative system driven by insider ownership seemed to be workable and 
supports the research in the field that suggests this is a desirable approach, but 
as indicated by this research, there are countless pitfalls related to efforts at 
democratic restructuring (Fullan, 1993; Glickman, 1992; Shindler, 1998). While 
this study found support for this being the “best” way to approach reform, it must 
accept that achieving meaningful reform in any such effort has a poor track 
record. Nonetheless, it appears that building in remedies to these pitfalls 
increases the chances for success. 

7. The need for change and a discomfort with the status quo shared by the teachers 
and the principal may be the most important catalysts for strengthening a 
school’s climate. 



Implications for Practice 

Given the conclusions above, those endeavoring to initial a school climate 

improvement effort might consider the following implications: 

1 . The analytic trait instrument provides an accurate mirror with which a school can 
reflect upon their own practice and performance. The use of the instrument 
within a participant-driven assessment and planning process provides a 
mechanism for process ownership and vision building. In contrast, the use an 
objective inventory scored externally may not provide the transparency 
necessary in the assessment process, nor the ownership necessary to facilitate 
vision building. 

2. The membership of the teacher leadership/assessment team is significant. The 
team that takes part in the assessment phase of the process will likely reach an 
understanding of the fundamental definitions at work and the implicit climate 
goals at the school. If those involved in a planning phase have not participated in 
this assessment phase they may not have an adequate foundation for their work. 
Therefore, it appears that having a team that will continue for both tasks is 
desirable. Likewise, selection of the initial team should consider the skills 
necessary for visionary leadership. 

3. A transparent examination of the construct of “good school climate” is desirable if 
not essential in any effort at fruitful democratic reform. A steering or vision team 
entrusted with the task of first assessing and then diagnosing their school’s 
needs, must posses the capacity to understand the broader implications of their 
work. 

4. The role of the school leadership, which in most cases involves the principal, is 
crucial. The leader must possess a vision of what the school can become, have a 
clarity about the school’s “goals,” and the ability to articulate that vision to the rest 
of the staff. No other person in the process is in the same position to create the 
necessary shared values and vision. 

5. The presence of vision and shared values appears to lead to an “intentional 
climate.” The absence of vision leads to what appears to be an “accidental 
climate.” It appears that little meaningful change can occur within a school no 
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matter how sound the instrument or the process if a sense of vision and a 
collective belief in the value of change is not cultivated. 

6. While the analytic trait assessment instrument can provide a team with a 
effective mirror in their efforts to reflect on the quality of their school’s climate, the 
mirror may be unwelcome. The reflection provided by the instrument may prove 
unsettling. This is especially true if the goal of the school tends to fall in the 
“domestication” category or if there is a misperception on the part of the principal 
about the true state of the school’s climate. 

7. The process of school climate examination should be undertaken with a full and 
deliberate understanding of its delicate nature. Trust among the leadership, the 
staff team, and outside consultants is critical. If the process is seen as a one that 
is “out to get someone” or as having an agenda it will have little chance of 
success. There are countless ways to derail the effort, but there may be only 
one way to produce meaningful reform. The effort must move slowly and 
deliberately with a broad base of participation and a sensitivity that improvement 
will in many respects imply that practices must change. Without collective trust, 
vision building and stakeholder commitment to the common interest of the school 
it is likely the effort will become at best, ineffectual; at worst, a source of strife. 
Furthermore, research suggests (Shindler, 1997) that if a first attempt at a reform 
such as this does not work, there are few if any second chances. 
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